Task: Determine The Test Strategy (MTP)

Method of operation

Determining a test strategy for a master test plan involves the following steps:

Determining test levels
Determining thoroughness of testing per characteristic/object part per test level

These steps are explained in greater detail below.

The product risk analysis and a first draft of the test strategy can often be combined in one single session. If this is impossible, the test manager creates a proposal for the test strategy after the product risk analysis, and discusses it with the client and several other stakeholders. A broadly supported result is achieved by creating a test strategy in consultation with these parties. This makes it easier to make effective decisions about the balance between thoroughness of testing, costs and time at a later stage.

Products

The test strategy, including explanation, laid down in the master test plan.
Summary description per test level, with at least the aim of the test and person or department responsible.

Techniques

Determining the strategy (as described the attached guidance.)

1. Determining test levels

This involves the test levels to which the master test plan relates. Good arrangements must be made over the test levels to be distinguished and the parties responsible for executing each test level. Many organisations distinguish a unit test (UT), unit integration test (UIT), system test (ST), sometimes a functional acceptance test (FAT), a users acceptance test (UAT), and a production acceptance test (PAT). The UT, UIT and ST usually are the responsibility of the development organisation. The future user organisation is responsible for the FAT and UAT. The future (technical) system management organisation is responsible for the PAT. It is preferred that reviewing also be part of the master test plan - which is what many organisations are already doing. This is the case in particular in organisations that do not create a separate quality or review plan.

Many organisations have a standard classifi cation of test levels. The test manager must examine in how far he must or can deviate from this. If the test levels have not yet been established or deviations are possible, in this step the test manager investigates the desired classification in test levels while taking account of the results of the product risk analysis.

When determining the test levels, arrangements are made about the aim and content of each test level.

2. Determining thoroughness of testing per characteristic/object part per test level

After the test levels included in the scope of the master test plan are determined, the specified characteristics/object parts from the product risk analysis are allocated to the test levels. This results in alignment between the various test levels that are executed within the project. Clearly, the relevant responsibilities and authorisations must not be neglected in this context. Often a type of standard division can be used based on the aim of the test levels (primarily functionality for ST, suitability and user-friendliness for UAT, etc).

First the characteristics and object parts from the PRA with their risk class (high, average, low) are set out against the test levels. It is then determined for each characteristic/object part combination how thoroughly it must be tested in which test level. A • symbol in a matrix shows whether a specific characteristic/object part is part of the test strategy of a test level. •• or even ••• shows that a relatively high level of attention must be devoted to the characteristic/object part in the relevant test level. Naturally a characteristic/object part can be included in more than one test level, but the depth of testing usually differs. If test design techniques are used, results of previous test levels can be reviewed in e.g. the acceptance test. On the basis of this, less thorough testing may be required in that test level.

Comments on the table above:

PRA-RC : Risk class (from PRA, where A=high risk, B=average risk, C=low risk)
Evaluation : Evaluation/review of the various intermediary products, such as requirements, functional design, technical design
Development test : Unit test + Unit integration test
ST : System Test
UAT : Users Acceptance Test
PAT : Production Acceptance Test
● : limited thoroughness of the dynamic test
●● : medium thoroughness of the dynamic test
●●● : high thoroughness of the dynamic test
S : Static testing - Checking and examining products without executing the software
I : Implicit testing - Including in another test type without creating specifically designed test cases.
<blank> : If a cell is bank, it means that the relevant test or evaluation level does not have to be concerned with the characteristic.